Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

What’s new with numbers? Sociological approaches to the study of quantification

Calculation and quantification have been critical features of modern societies, closely linked to science, markets, and administration. In the past thirty years, the pace, purpose, and scope of quantification have greatly expanded, and there has been a corresponding increase in scholarship on quantification. We offer an assessment of the widely dispersed literature on quantification across four domains where quantification and quantification scholarship have particularly flourished: administration, democratic rule, economics, and personal life. In doing so, we seek to stimulate more cross-disciplinary debate and exchange. We caution against unifying accounts of quantification and highlight the importance of tracking quantification across different sites in order to appreciate its essential ambiguity and conduct more systematic investigations of interactions between different quantification regimes

The rise of ‘responsible metrics’ as a professional reform movement: a collective action frames perspective

Recent years have seen a rise in awareness around “responsible metrics” and calls for research assessment reforms internationally. Yet within the field of quantitative science studies and in research policy contexts, concerns about the limitations of evaluative bibliometrics are almost as old as the tools themselves. Given that many of the concerns articulated in recent reform movements go back decades, why has momentum for change grown only in the past ten years? In this paper, we draw on analytical insights from the sociology of social movements on collective action frames to chart the emergence, development, and expansion of “responsible metrics” as a professional reform movement. Through reviewing important texts that have shaped reform efforts, we argue that hitherto, three framings have underpinned the responsible metrics reform agenda: the metrics scepticism framing, the professional-expert framing, and the reflexivity framing. We suggest that while these three framings have co-existed within the responsible metrics movement to date, the “truce” negotiated between these framings may not last indefinitely, especially as the responsible metrics movement extends into wider research assessment reform movements.</p

Quantified academic selves: The gamification of research through social networking services

Introduction Our study critically engages with techniques of self-quantification in contemporary academia, by demonstrating how social networking services enact research and scholarly communication as a ‘game’. Method The empirical part of the study involves an analysis of two leading platforms: Impactstory and ResearchGate. Observed qualities of these platforms will be analyzed in detail with concrete examples of gaming features in focus. Subsequently, we relate the development of these digital platforms to a broader ‘quantified self movement’. Special attention will also be paid to how these platforms contribute to a general quantification of the academic (authorial) self. Theory Theoretically we relate the ‘gamification’ of research to neoliberal ideas about markets and competition. Our analysis then extends to long-standing and fundamental ideas about self-betterment expressed in the philosophy of Peter Sloterdijk. Findings Our study shows how social networking services, such as ResearchGate and Impactstory, enact researchers as ‘entrepreneurs of themselves’ in a marketplace of ideas, and the quantification of scholarly reputation to a single number plays an important role in this process. Moreover, the technologies that afford these types of quantifiable interactions affect the ‘unfolding ontology’ of algorithmic academic identities. Conclusion The gamification of quantified academic selves intensifies the competitive nature of scholarship, it commodifies academic outputs and it might lead to goal displacement and cheating. However, self-quantification might also serve as a liberating and empowering activity for the individual researcher as alternative measures of impact and productivity are provided by these platforms.</p

Quantified academic selves: The gamification of research through social networking services

Introduction Our study critically engages with techniques of self-quantification in contemporary academia, by demonstrating how social networking services enact research and scholarly communication as a ‘game’. Method The empirical part of the study involves an analysis of two leading platforms: Impactstory and ResearchGate. Observed qualities of these platforms will be analyzed in detail with concrete examples of gaming features in focus. Subsequently, we relate the development of these digital platforms to a broader ‘quantified self movement’. Special attention will also be paid to how these platforms contribute to a general quantification of the academic (authorial) self. Theory Theoretically we relate the ‘gamification’ of research to neoliberal ideas about markets and competition. Our analysis then extends to long-standing and fundamental ideas about self-betterment expressed in the philosophy of Peter Sloterdijk. Findings Our study shows how social networking services, such as ResearchGate and Impactstory, enact researchers as ‘entrepreneurs of themselves’ in a marketplace of ideas, and the quantification of scholarly reputation to a single number plays an important role in this process. Moreover, the technologies that afford these types of quantifiable interactions affect the ‘unfolding ontology’ of algorithmic academic identities. Conclusion The gamification of quantified academic selves intensifies the competitive nature of scholarship, it commodifies academic outputs and it might lead to goal displacement and cheating. However, self-quantification might also serve as a liberating and empowering activity for the individual researcher as alternative measures of impact and productivity are provided by these platforms.</p